FACILITATING COMPUTER-AIDED DETECTION, COMPARISON 
AND/OR DISPLAY OF MEDICAL IMAGES 



CROSS-REFERENCE TO RELATED APPLICATION 

5 This application is a continuation-in-part of Application Serial No. 09/992,059, 

filed November 21, 2001 for "A Method and Apparatus for an Improved Computer Aided 
Diagnosis System," and Application Serial No. 10/079,327 filed February 19, 2002, for 
"A Method and Apparatus for Expanding the Use of Existing Computer- Aided Detection 
Code" which applications are incorporated herein by reference. 
10 The subject matter of the present application is related to commonly assigned 

Application Serial No. , of P. Snoeren and N. Karssemeijer filed concurrently 

herewith, for "Model-Based Grayscale Registration of Medical Images". This application 
is incorporated herein by reference. 

15 FIELD OF THE INVENTION 

The present invention relates to facilitating computer-aided detection, comparison 
and/or display of medical images. A particularly useful application of the invention is in 
the field of radiographic mammography and the invention will be described in detail in 
that context. The invention may also be practiced in numerous other contexts. 

20 

BACKGROUND OF THE INVENTION 

Mammography is a specialized form of radiography designed to detect the subtle 
changes in x-ray attenuation that are caused by cancerous tissue when x-rays irradiate the 
human breast. While there is presently no means for preventing breast cancer, early 

25 detection of the disease prolongs life expectancy and decreases the likelihood of the need 
for a total mastectomy. Radiographic mammography is currently the most common 
method of detecting and analyzing breast lesions. The American Medical Association, 
The American Cancer Society, and the American College of Radiology recommend 
yearly mammograms for women beginning at age 40. 

30 Particular features in mammograms that are indicative of breast cancer include 

spiculated, or stellar- shaped, lesions and microcalcifications. While both features have a 
relatively high probability of being malignant, both features are also difficult to detect. 
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To detect any feature at all, the x-ray attenuation of that feature must differ appreciably 
from that of its environment. In the case of cancerous tissue, there is very little difference 
in attenuation between cancerous tissue and the glandular tissue in which it is found at x- 
ray energies above about 35 keV and a difference of only about 10% at x-ray energies of 
5 about 20 keV. Detection of spiculated masses is further complicated by the presence in 
typical mammograms of myriad lines corresponding to fibrous breast tissue. In the case 
of microcalcifications, while they are almost radiopaque, they are usually very small and 
faint in a mammogram and it is very difficult to distinguish cancerous microcalcifications 
from numerous other artifacts that are of similar size and appearance. 

10 A typical analog or film-based mammography system 100 is shown in Figure 1 . 

The system comprises an x-ray tube 110, upper and lower compression paddles 130,135 
an anti-scatter grid 140, a screen-film unit 150, and a phototimer detector 160, all of 
which are mounted on a frame 170. The x-ray tube comprises a cathode 1 12, an anode 
114 that is mounted on a shaft 116 and rotated by a motor 1 1 8, a tube port 120, a filter 

15 122 and a collimator 124. The screen-film unit includes an x-ray film 152 and a 

fluorescent screen 154. Phototimer detector 160 measures the total exposure. A control 
system (not shown) controls the operation of the x-ray tube including the peak voltage 
and tube current and terminates operation when a desired exposure as measured by 
detector 160 has been reached. Advantageously, the control system also includes a 

20 subsystem for measuring the space between compression paddles 130, 135 and therefore 
the thickness of the breast. 

To make a mammogram, a patient's breast is compressed between a lower surface 
132 of the upper compression paddle 130 and an upper surface 137 of the lower 
compression paddle; and the x-ray tube is turned on. Motor 118 rotates anode 1 14 while 

25 high energy electrons bombard the rotating anode causing the anode to emit x-rays. 

Some of the x-rays are emitted through tube port 120 in the direction of the breast located 
between the two compression paddles. The x-rays are band pass filtered by filter 122 to 
eliminate x-rays of especially high or low energies and are collimated by collimator 124 
to eliminate those x-rays traveling in unwanted directions. The remaining x-rays pass 

30 through the breast where they are selectively attenuated and are incident on the anti- 
scatter grid 140. The x-rays that pass through the anti-scatter grid then pass through the 
x-ray film with little interaction with the film and are incident on the fluorescent screen 
154. The x-rays interact with the fluorescent material in the screen, causing this material 
to emit radiation that interacts with the x-ray film to produce the x-ray image. Some of 

2 

CAl: 341700.2 



the x-rays also pass through the fluorescent screen and are incident on the phototimer 
detector 160. 

A variety of choices are available in the physical properties of these systems. The 
optimal x-ray energy range for these systems is about 17 to 23 keV. Within this range, 
5 molybdenum has characteristic x-ray peaks at 17.5 and 19.6 keV and rhodium has such 
peaks at 20.2 and 22.7 keV; and anodes made of one or the other of these elements are 
widely used. Typically, the tube port is made of beryllium which has low attenuation. 
The filters are typically made of the same material as the anode but a rhodium filter is 
also used with a molybdenum anode for imaging thicker and denser breasts. 

10 Ideally, the mammography system forms on film 152 a projection image of the 

attenuation of x-ray photons that traveled on straight lines from the anode through the 
breast to the film. However, the distribution of photons incident per unit area on the film 
is not uniform. Absorption of photons within the anode creates a "heel effect" as a result 
of which the area of the film directly under the anode will receive significantly fewer 

15 photons per unit area than the area of the film under the cathode. 

The photons may also be redirected by Compton or Rayleigh scattering and arrive 
at the film from many different angles other than angles corresponding to a straight line 
from the anode. Such scattered photons reduce the contrast in the mammogram. The 
amount of scatter in mammography varies with increasing breast thickness and breast 

20 area. For a typical 5 cm-thick breast, the contrast reduction due to scatter is on the order 
of 33%. To reject scatter, parallel linear grids with a grid ratio of 4:1 to 5:1 are 
commonly used. While the film is exposed, the grids are oscillated over a short distance 
to blur the grid lines. A cellular grid structure is also used in some systems to reject 
scatter in two dimensions. 

25 Various screen film systems are available from suppliers such as Agfa, Fuji and 

Kodak. All of these systems have a single gadolinium oxysulfide phosphor screen which 
produces green light and a green-sensitive single emulsion film. A variety of different 
speed films are available; and the characteristic curves of optical density versus exposure 
of these films can be quite different. Typical characteristic curves for two screen-film 

30 systems are shown in Fig. 2. Of particular note, the contrast in an image is a function of 
the slope of the characteristic curve. 

Recently, mammography systems have become available that use digital detectors 
in place of a screen-film system. These systems produce digital mammograms without 
the intervening steps of processing a film and then digitizing it. The digital systems 
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introduce considerably more variability in the process conditions. In addition to replacing 
the screen film combination, they also use different anode targets (typically, tungsten) and 
possibly other filters. 

In addition to variations in the physical properties of the mammography system, 
5 numerous operational parameters are within the control of the operator. These include the 
x-ray energy, typically specified in peak voltage (kVp), the exposure, typically specified 
in milli-Ampere-seconds (mAs), and the processing of the x-ray film. Another factor that 
clearly affects the optical density recorded on the film is the thickness of the breast being 
x-rayed and its density (or proportion of glandular tissue to total breast thickness). To a 

10 limited degree, the thickness of the breast being x-rayed can be controlled by the operator 
by adjusting the pressure exerted by the upper compression paddle. 

Despite the large number of physical and operational variables that exist in 
mammography systems, these differences are not an issue when reading a single set of 
mammograms taken at the same time under the same conditions. In reading the 

15 mammograms, the radiologist's attention is focused on the relative difference between 
adjacent regions of the mammogram; and since the mammogram was made under one set 
of conditions; these conditions have little effect on relative differences. However, the 
radiologist frequently wants to compare one set of mammograms with another set of 
mammograms, for example, a set of mammograms taken the previous year for the same 

20 person. In this case, there may be substantial differences between the two sets, for 

example, because they were taken on different systems, or recorded on different films, or 
taken with x-rays of different energy, or for different exposures. Needless to say, there 
are also substantial differences between film-based mammograms and digital 
mammograms. Similar issues arise in analyzing mammograms of different persons. 

25 Efforts have been made to address these problems by abstracting out at least some 

of the differences attributable to the physical and operational variables. In 
Mammographic Image Analysis (Kluver 1999), Ralph Highnam and Michael Brady 
describe how to correct and remove the effects of x-ray scatter, x-ray energy (kVp), 
exposure (mAs) and breast thickness. See also, their PCT application PCT/GB00/00617 

30 filed February 21, 2000 and published as publication WO 00/52641 on September 8, 
2000, which is incorporated herein by reference. The result is a completely physical 
description of the breast in terms of thickness and type of material - fat or glandular 
tissue. Their interest is in the glandular or interesting tissue and they call this description 
Hint, which is expressed in units of centimeters. A complete physical description of the 
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breast would require a combination of H in t and either the total breast thickness, H to t„ or the 
fat thickness, H fat , where H tot = H int + H fat . 

However, the H int image is very difficult for the radiologist to interpret since it is 
radically different from the conventional image the radiologist has been trained to 
5 interpret. Moreover, the computations needed to produce the H int image are extensive and 
require substantial amounts of processing time. 

Another approach is described in USP 6,516,045 for "Device and Method for 
Determining Proportions of Body Materials", which is incorporated herein by reference. 
As shown in Fig. 3, in this technique, two right-angled wedge-shaped reference materials 
10 302, 304 are positioned alongside the breast 306 between the compression paddles. One 
wedge has the attenuation characteristics of fat. The other wedge has the attenuation 
characteristics of glandular tissue. The base of each wedge is the thickness of the breast. 
As a result, when the mammogram is formed, an image is created of the wedges as well 
as the breast and the optical density of the image of the wedges ranges continuously from 
15 a value corresponding to 100% fat to 100% glandular tissue. Since the shape of the 
wedges is known, the optical density of each point in the image of the wedges can be 
associated with a specific percentage of fat and glandular tissue. Then by matching each 
pixel of the breast image with the pixels of the wedge having the same optical density, the 
percentage of fat and glandular tissue at that pixel in the breast image can be determined. 
20 Still another approach is described in the co-pending applications Serial No. 

09/992, 059 for "A Method and Apparatus for an Improved Computer Aided Diagnosis 
System," and Serial No. 10/079,327 for "A Method and Apparatus for Expanding the Use 
of Existing Computer- Aided Detection Code" of which the present application is a 
continuation-in-part. In those applications, various normalization techniques are 
25 described to remove the differences caused by different detectors. In particular, the 

applications describe a variety of techniques for equalizing the contrast response in which 
analytic expressions or "look-up" tables are developed that convert the response 
measured by one system to what the response would be if measured by another system. 
While these techniques facilitate the analysis and comparison of mammograms made 
30 using different detectors, they do not address differences arising from different exposure 
parameters or differences in breast thickness. 

Another area in which it would be advantageous to compensate for differences 
arising from different exposure parameters or differences in breast thickness is in the 
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development of algorithms for computer aided detection and diagnosis of abnormalities in 
medical images such as mammograms. 

The algorithms that are presently used are heavily dependent on the training of the 
algorithms using groups of mammograms. See, for example, USP 5,491,627 to Zhang et 
5 al., USP 6,075,879 to Roehrig et aL, and the above-referenced Serial No. 10/079,327 for 
"A Method and Apparatus for Expanding the Use of Existing Computer- Aided Detection 
Code." At present, to obtain a sufficient number of mammograms for training purposes, 
the set of training mammograms includes mammograms formed on different 
mammographic systems. As a result, the performance of the computer aided detection 
10 system is not as great as it would be if the training had been performed on the same sized 
set of mammograms made on a single system. It would be advantageous to be able to 
train the detection algorithm using larger sets of mammograms made on effectively the 
same mammographic system. 

15 SUMMARY OF THE INVENTION 

The present invention is methods for processing medical images so as to remove 
certain effects of the physical characteristics of the object being imaged and/or of the 
apparatus used to form the images. The invention further provides for the formation of a 
standardized image from the processed image and for the use of the standardized image or 
20 the processed image in the training of computer-aided detection/diagnosis algorithms. 
These algorithms may then be used to detect abnormalities in other standardized or 
processed images derived from any of a variety of image acquisition systems. 

BRIEF DESCRIPTION OF THE DRAWING 

25 These and other objects, features and advantages of the invention will be more 

readily apparent from the following detailed description of the invention in which: 

Fig. 1 is a schematic illustration of a prior art analog mammography system; 

Fig. 2 is a depiction of the characteristic response of two radiographic 
mammography films; 

30 Fig. 3 is a schematic illustration of a prior art modification to a mammography 

system; 

Fig. 4 is a flowchart depicting a first embodiment of the invention; 
Fig. 5 is a flowchart depicting a second embodiment of the invention; 
Fig. 6 is a flowchart depicting a third embodiment of the invention. 
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Figs. 7A and 7B are illustrative mammograms, one formed by a conventional 
system and the other formed in accordance with an embodiment of the invention. 

DETAILED DESCRIPTION OF THE INVENTION 

5 Fig. 4 is a flowchart depicting a first embodiment of the invention. At step 410 a 

mammogram is formed using either an analog mammography system such as that shown 
in Fig. 1 or a digital mammography system. If an analog system is used in step 410, the 
resulting mammogram film is then scanned at step 415 to convert the analog image into a 
digitized image. At step 420, the digital image formed in step 410 or the digitized imaged 

10 formed in step 415 is processed to remove at least one and preferably substantially all 
distinguishing effects related to the physical characteristics of the first mammography 
system and its operating parameters. 

These effects may include effects arising from physical characteristics of the 
mammography system such as anode material, source to image distance, anti-scatter grid 

15 geometry, film characteristics, and screen film system. They may also include operating 
characteristics of the mammography system that may vary from patient to patient even 
when using the same mammography system such as x-ray energy, and magnitude and 
duration of exposure as well as physical differences between patients such as thickness 
and density of the breast being imaged. In addition, processing preferably is performed to 

20 measure the relative fat content of the breast being imaged. 

The processing of the digital or digitized mammogram is preferably performed by 
a computer. Preferably, the processing performed is essentially the same as that detailed 
by Highnam and Brady in their book Mammographic Image Analysis and in their PCT 
patent application PCT/GB00/00617. As indicated above, extensive processing is 

25 necessary to remove the effects of the mammography system as well as to remove the 
effects of the fat content of the breast which is not interesting to the radiologist. 

The result of the processing is an image representative of the physical 
characteristics of the breast that was imaged by the first mammography system and, in 
particular, representative of its glandular (non-fatty) tissue content. This image is referred 

30 to by Highnam and Brady as the H int (or interesting) image and comprises a two- 
dimensional array of numerical values representative of the thickness of glandular tissue 
at each point in the array. 

At step 430, the processed image is converted from the Hi nt image to a standard- 
form mammogram by calculating what the original mammogram would have looked like 
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had it been made on a standard- form mammography system having a standard set of 
physical characteristics and a standard set of operating parameters. These calculations are 
essentially the reverse of the calculations used to form the processed image but using in 
this case the physical characteristics and operating parameters of the standard-form 
5 mammography system. For example, for the standard- form system, the x-ray energy 
preferably is set to 25 kVp, the exposure to 100 mAs, and the breast thickness to 5 cm. 
These parameters are preferred in view of the energy dependence of breast x-ray 
attenuation. Alternatively, the x-ray energy may be set at a value within the range 25-28 
kVp and the exposure at a value within the range 20-200 mAs, and the thickness to a 

10 value within 4-6 centimeters. The processing required to form the standard-form image is 
extensive and preferably is performed by a computer. 

At step 440, the standard form mammogram is then stored. 
In like fashion, other digital or digitized mammograms formed by the same 
mammography system or by different mammography systems are similarly processed by 

15 the system of the present invention to form processed images from which have been 

removed the same effects related to the physical characteristics and operating parameters 
of the mammography system as well as to the fat content of the breast; and the processed 
images are converted to standard-form mammograms and stored. 

The standard-form mammograms are then available to be compared at step 450. 

20 In all cases, the standard- form mammograms are free of the differences that otherwise 

would be present arising from the use of different mammography systems and/or different 
operating parameters in the formation of the mammogram. In the case where the 
standard-form mammograms are images of the same breast, the images will be essentially 
the same except for any physiological changes that have occurred in the breast between 

25 the time the first mammogram was taken and the time any later mammogram was taken. 

Comparisons may be made using a variety of modalities such as computer work 
stations, film display systems, and images printed on paper. Again, there are considerable 
differences in the display characteristics of the different modalities and significant 
differences in the display characteristics of different suppliers' products in the same 

30 modality. Of particular interest are differences in brightness and contrast in the displayed 
images. Advantageously, these differences can be controlled to meet the needs of the 
user. For example, analytic expressions and/or look-up tables can be developed that 
relate pixel values of standard-form images stored at step 440 to levels of brightness on a 
particular manufacturer's work station, to levels of optical density in a film or to gray 
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scale values in an image printed on paper by a laser printer. These expressions or look-up 
tables are then used at step 450 to convert the standard form images to images having the 
desired brightness and contrast in the display modality of choice. 

To understand how to obtain a purely physical description of the breast such as the 
5 Hi m image, a simplified example of computing Hi nt is presented here for three hypothetical 
locations in a breast, including a first location that is 100% fat, a second location that is 
50% fat and 50% glandular, and a third location that is 100% glandular. We will assume 
the breast is uniformly exposed to mono-energetic x-rays in a device such as shown in Fig 
1. We will assume further that the compression thickness H to t is uniform - i.e. the 
10 compression paddles are parallel. Actual computations to account for the true x-ray 
spectrum, and to correct effects such as tilt in the compression paddles, will be more 
complex than shown here, but the description below is adequate to describe the principle 
of how to derive the H int image from the exposure of the actual breast. 

The intensity of x-ray penetrating material is given by Beer's law: 

15 

lout = Iin exp(-|aAh) 

where |j. is the attenuation of x-rays through matter, and Ah is the thickness of the 
material. Let us suppose the attenuation of fat is given by jaf at and the attenuation of 
20 glandular tissue is tissue- In reality the attenuation is also a function of energy of the x- 
ray, but here we will assume that the x-rays are mono-energetic and the attenuation can 
then be represented with a single number jlx. 

Then the intensity after passing through H tot of a fatty section, an equal proportion 
of fat and tissue, and a pure tissue regions will be: 

25 

I in exp(-ja fat H tot ) 

I in exp(-Of at +Mtissue )H tot / 2 ) 
Iin eXp(-|J,tissue Htot ) 

30 We assume that the exposure parameters are somehow available so that Ii n (in 

units of mAs or milliAmp-sec) is known, the values for attenuation jx in fat and tissue are 
known, and the thickness H tot has been measured. If the mammogram has been acquired 
on film/screen, these parameters are written onto the patient labels which appear on the 
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margins of the films. Co-pending patent application Serial No. 10/142,704 for "A 
Method and Apparatus to Associate User Data from a Radiograph" describes an 
automated technique for obtaining that information using optical character recognition 
(OCR). This application is incorporated herein by reference. If the mammogram has been 
5 acquired on a digital detector, the exposure parameters are entered into fields in the 
standard DICOM header which is created for each image. 

The equations above relate the intensity of x-rays at the detector to the exposure 
and material characteristics. To actually perform any actual operations in the computer, 
we must also know how the intensity maps into pixel value in the computer image. This 
10 mapping is given by the equation 

PV = Kln (I/F), 

which is the linear portion of the film response shown in the characteristic curves of Fig. 
15 2. For purposes of this discussion, without loss of generality, let us ignore the constants K 
and F. Then the pixel values for the image of the fatty section, an equal proportion of fat 
and tissue and a pure tissue region are: 

PV^lnO-Hfat H to t 

20 PV 2 = ln(I ) - (Ifat H tot 12 - tissue Htot 12 
PV 3 = ln(I ) - tissue H tot 

Or, subtracting the constant factor ln(I) - ji fa t H to t , and defining \i& = tissue - Ktfat , we can 
express the three pixel values as: 

25 

PV1 =0 

PV2 - -n A H tot 12 
PV3 = -Ki A H tot 

30 which are in the form PV = |Li in t H in t, where H int is 0 in the portion that is 100% fat, l A of 
Htot in the region with 50% tissue, and 1 H tot in the region with 100% tissue. Thus, the 
pixel values are a function of the thickness of the glandular or interesting tissue in the 
breast. This is essentially the "Hint" image as used by Highnam and Brady, and provides 

10 

CA I: 341700.2 



a representation of the mammogram which is free of the exposure parameters kVp, mAs, 
and compression thickness. 

The standard image formed at step 430 differs from the H im image described in the 
above in that it will have the appearance of an x-ray mammogram with a known, 
"standard" thickness. Without loss of generality, we choose 5cm as the "standard" 
thickness because this is a thickness that is quite common in actual mammograms. 
Further, we assume the mono-energetic x-ray energy to be 25 kVp, and the intensity to be 
1= lOOmAs. 

We convert the H int image to the standard-form "normal" image as follows. The 
minimum pixel value of this particular H in t image is 0, which is the assigned value for 
pure fatty region of thickness 5cm. Given a value of attenuation for fat at 25 kVp to be 
Hfat(25> the H^t image can be converted to a standard form image by adding 
lnl00exp[(ifat(25) * (5 - H int )] to each pixel value. The mapping of the H int image to the 
standard form image expressed by this exponential can be implemented in a look up table 
(LUT). If it is desired to represent the image as it would appear with other exposure 
parameters, i.e., other H to t, intensity I, and attenuation corresponding to another kVp, this 
mapping can be implemented by : 

PVnew = PVnorm + [ln(I ne w) - Ufat (kVp) T] , which can be implemented in another LUT. 

It remains to show how to map the appearance into the appearance of the standard 
form image of any given detector. 

Fig. 2 shows the characteristic curves of two film brands used in mammography - 
Kodak Min-R2000 and Agfa HDR. These can be obtained from the manufacturers, or 
measured empirically by, for example, step wedges. The mapping from pixel value 
obtained from Agfa, for example, to Kodak Min-R2000 can be obtained directly from the 
two curves in Fig. 2, which can be implemented easily in a LUT. For example, an OD of 
1 .6 on Agfa would be mapped to approximately 2.4 to convert to a "Kodak" appearance. 
Similarly, a film/screen image from e.g., Kodak, can be converted to a digital detector 
which also has a characteristic response. Conversion from all types of mammographic 
image into all other types can be implemented in look up tables. Hence, we can in 
principle convert mammographic images from all sources into the common form and 
from the common form into any display form thus removing dependence on detector type. 
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Fig. 5 is a flow-chart depicting a second embodiment of the invention. At step 
510 a mammogram is formed using either an analog mammography system or a digital 
mammography system. Two wedge-shaped materials as illustrated in Fig. 3 are irradiated 
at the same time as the breast and their images are formed alongside the breast on the 
5 mammogram. If an analog mammography system is used in step 510, the mammogram 
film is then digitized at step 515 by a scanner to produce a digitized image of pixel 
values. 

Whether formed directly by a digital mammography system in step 510 or by 
digitizing a file in step 515, the portion of the resulting image that is the image of the 

10 wedges will have pixel values ranging from a first value corresponding to 100% fatty 

tissue to a second value corresponding to 100% glandular tissue; and each pixel value in 
the image of the breast between the first and second values will be associable with a 
specific percentage of fatty and glandular tissue. 

At step 520, a look-up table is created associating each pixel value in the image of 

1 5 the wedge with a specific percentage of fat and glandular tissue; and the pixel values in 
the digital or digitized image of the breast are converted to an image representative of the 
physical characteristics of the breast (i.e., percentage of fatty and glandular tissue) by 
using the table to convert pixel values in the breast image to percentages. 

The physical image may then be used in the same fashion as the processed image 

20 produced at step 420 in the first embodiment. In particular, the physical image may be 
converted at step 530 to a standard form mammogram using a standard set of exposure 
parameters such as a peak energy level of 25 kVp and an exposure of 100 mAs for a 
breast having a thickness of 5 cm. The standard form mammogram may then be stored at 
step 540. Additional standard form mammograms may likewise be made and stored and 

25 any of these mammograms may later be retrieved at step 550 for display and comparison 
on suitable display modalities. 

In addition to facilitating the comparison of mammograms, the present invention 
also facilitates the analysis of mammograms in computer aided detection and diagnosis 
systems. As indicated above, the algorithms used in such systems are typically trained on 

30 a large number of mammograms and this typically means that the mammograms were 
made on more than one mammography system. 

In accordance with another embodiment of the present invention, algorithms are 
trained either on standard- form images or on physical images and mammograms are 
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